---
sidebar_position: 10
description: How to configure an AWS S3 exporter.
---
import { integrations } from "@site/data/integrations";
import {Mandatory, NotMandatory} from "@site/src/components/checks/checks";
export const exporterName = 'AWS S3'
export const info = integrations.exporters.find((r) => r.name === exporterName)

# AWS S3

## Overview
{info.longDescription ?? info.description}

Everytime the `FlushInterval` or `MaxEventInMemory` is reached a new file will be added to {exporterName}.

:::info
If for some reason the {exporterName} upload fails, we will keep the data in memory and retry to add the next time we reach `FlushInterval` or `MaxEventInMemory`.
:::

![export in S3 screenshot](/docs/data_collection/s3-exporter.png)

## Configure the relay proxy

To configure your relay proxy to use the {exporterName} exporter, you need to add the following
configuration to your relay proxy configuration file:

```yaml title="goff-proxy.yaml"
# ...
exporters:
  - kind: s3
    bucket: evaluation-data-bucket
# ...
```

:::info
If you are using the S3 exporter, the easiest way to provide credentials is to set environment variables. It will be used by GO Feature Flag to identify to your S3 bucket.

```shell
export AWS_SECRET_ACCESS_KEY=xxxx
export AWS_ACCESS_KEY_ID=xxx
export AWS_DEFAULT_REGION=eu-west-1
```
:::

| Field name                |    Mandatory     | Type   | Default                                                                                                                            | Description                                                                                                                                                                                                                                                                                                                                                                           |
|---------------------------|:----------------:|--------|------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `kind`                    |  <Mandatory />   | string | **none**                                                                                                                           | **(mandatory)** Value should be **`s3`**.<br/>_This field is mandatory and describes which retriever you are using._                                                                                                                                                                                                                                                                  |
| `bucket`                  |  <Mandatory />   | string | **none**                                                                                                                           | **(mandatory)** Name of your S3 Bucket.                                                                                                                                                                                                                                                                                                                                               |
| `flushInterval`           | <NotMandatory /> | int    | `60000`                                                                                                                            | The interval in millisecond between 2 calls to the webhook _(if the `maxEventInMemory` is reached before the flushInterval we will call the exporter before)_.                                                                                                                                                                                                                        |
| `maxEventInMemory`        | <NotMandatory /> | int    | `100000`                                                                                                                           | If we hit that limit we will call the exporter.                                                                                                                                                                                                                                                                                                                                       |
| `format`                  | <NotMandatory /> | string | `JSON`                                                                                                                             | Format is the output format you want in your exported file. Available format: `JSON`, `CSV`, `Parquet`.                                                                                                                                                                                                                                                                               |
| `filename`                | <NotMandatory /> | string | `flag-variation-{{ .Hostname}}-{{ .Timestamp}}.{{ .Format}}`                                                                       | You can use a config template to define the name of your exported files. Available replacements are `{{ .Hostname}}`, `{{ .Timestamp}}` and `{{ .Format}`                                                                                                                                                                                                                             |
| `csvTemplate`             | <NotMandatory /> | string | `{{ .Kind}};{{ .ContextKind}};{{ .UserKey}};{{ .CreationDate}};{{ .Key}};{{ .Variation}};{{ .Value}};{{ .Default}};{{ .Source}}\n` | CsvTemplate is used if your output format is CSV.<br/>This field will be ignored if you are using format other than CSV.<br/>You can decide which fields you want in your CSV line with a go-template syntax, please check [`exporter/feature_event.go`](https://github.com/thomaspoignant/go-feature-flag/blob/main/exporter/feature_event.go) to see what are the fields available. |`
| `path`                    | <NotMandatory /> | string | **bucket root level**                                                                                                              | The location of the directory in S3.                                                                                                                                                                                                                                                                                                                                                  |
| `parquetCompressionCodec` | <NotMandatory /> | string | `SNAPPY`                                                                                                                           | ParquetCompressionCodec is the parquet compression codec for better space efficiency. [Available options](https://github.com/apache/parquet-format/blob/master/Compression.md)                                                                                                                                                                                                        |`

### Compatibility with S3-Compatible Solutions

The S3 retriever is compatible with S3-compatible on-premises solutions such as `MinIO`.  
To use an S3-compatible storage provider, ensure that you configure the following parameters:

- **Endpoint URL**: Provide the endpoint URL of your S3-compatible solution.
- **Access Credentials**: Use your S3-compatible solution's access key and secret key.
- **Bucket and Key**: Specify the bucket name and the key where your configuration file is stored.

:::warning
The AWS SDK requires a region to be set, even if you are using an S3-compatible solution.
```shell
export AWS_DEFAULT_REGION=xxx
````
:::

:::info
If you get spammed with
```shell
SDK 2025/04/25 13:07:36 WARN Response has no supported checksum.
````
Not validating response payload. you might want to set `export AWS_RESPONSE_CHECKSUM_VALIDATION=when_required` as well.
:::

## Configure the GO Module
To configure your GO module to use the {exporterName} exporter, you need to add the following
configuration to your `ffclient.Config{}` object:

:::info
The [**S3 Exporter v2**](https://pkg.go.dev/github.com/thomaspoignant/go-feature-flag/exporter/s3exporterv2) will use the [`aws-sdk-go-v2`](https://github.com/aws/aws-sdk-go-v2) to store your evaluation data in an S3 bucket.

The [**S3 Retriever v1**](https://pkg.go.dev/github.com/thomaspoignant/go-feature-flag/exporter/s3exporter) will use the deprecated [`aws-sdk-go`](https://github.com/aws/aws-sdk-go) to access your flag in an S3 bucket.

[AWS has announce end-of-support for AWS SDK for Go v1](https://aws.amazon.com/blogs/developer/announcing-end-of-support-for-aws-sdk-for-go-v1-on-july-31-2025/), and it's recommended to migrate from S3 Retriever v1 to v2.
:::

```go title="example.go"
awsConfig, _ := config.LoadDefaultConfig(context.Background())
config := ffclient.Config{
    // ...
   DataExporter: ffclient.DataExporter{
        // ...
        Exporter: &s3exporterv2.Exporter{
            Format: "csv",
            FileName: "flag-variation-{{ .Hostname}}-{{ .Timestamp}}.{{ .Format}}",
            CsvTemplate: "{{ .Kind}};{{ .ContextKind}};{{ .UserKey}};{{ .CreationDate}};{{ .Key}};{{ .Variation}};{{ .Value}};{{ .Default}};{{ .Source}}\n",
            Bucket:    "my-bucket",
            S3Path:    "/go-feature-flag/variations/",
            Filename:  "flag-variation-{{ .Timestamp}}.{{ .Format}}",
            AwsConfig: &awsConfig,
        },
    },
    // ...
}
err := ffclient.Init(config)
defer ffclient.Close()
```

| Field name                |    Mandatory     | Default                                                                                                                            | Description                                                                                                                                                                                                                                                                                                                                                                           |
|---------------------------|:----------------:|------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `Bucket`                  |  <Mandatory />   | **none**                                                                                                                           | Name of your S3 Bucket.                                                                                                                                                                                                                                                                                                                                                               |
| `AwsConfig`               |  <Mandatory />   | **none**                                                                                                                           | An instance of `aws.Config` that configures your access to AWS *(see [this documentation for more info](https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/))*.                                                                                                                                                                                                                 |
| `FlushInterval`           | <NotMandatory /> | 60000`                                                                                                                             | The interval in millisecond between 2 calls to the webhook _(if the `maxEventInMemory` is reached before the flushInterval we will call the exporter before)_.                                                                                                                                                                                                                        |
| `MaxEventInMemory`        | <NotMandatory /> | `100000`                                                                                                                           | If we hit that limit we will call the exporter.                                                                                                                                                                                                                                                                                                                                       |
| `Format`                  | <NotMandatory /> | `JSON`                                                                                                                             | Format is the output format you want in your exported file. Available format: `JSON`, `CSV`, `Parquet`.                                                                                                                                                                                                                                                                               |
| `Filename`                | <NotMandatory /> | `flag-variation-{{ .Hostname}}-{{ .Timestamp}}.{{ .Format}}`                                                                       | You can use a config template to define the name of your exported files. Available replacements are `{{ .Hostname}}`, `{{ .Timestamp}}` and `{{ .Format}`                                                                                                                                                                                                                             |
| `CsvTemplate`             | <NotMandatory /> | `{{ .Kind}};{{ .ContextKind}};{{ .UserKey}};{{ .CreationDate}};{{ .Key}};{{ .Variation}};{{ .Value}};{{ .Default}};{{ .Source}}\n` | CsvTemplate is used if your output format is CSV.<br/>This field will be ignored if you are using format other than CSV.<br/>You can decide which fields you want in your CSV line with a go-template syntax, please check [`exporter/feature_event.go`](https://github.com/thomaspoignant/go-feature-flag/blob/main/exporter/feature_event.go) to see what are the fields available. |`
| `Path`                    | <NotMandatory /> | **bucket root level**                                                                                                              | The location of the directory in S3.                                                                                                                                                                                                                                                                                                                                                  |
| `ParquetCompressionCodec` | <NotMandatory /> | `SNAPPY`                                                                                                                           | ParquetCompressionCodec is the parquet compression codec for better space efficiency. [Available options](https://github.com/apache/parquet-format/blob/master/Compression.md)                                                                                                                                                                                                        |`
